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Education Evolving is a Minnesota-based, nonprofit, nonpartisan organization focused on improving 
educational opportunities and outcomes for all students, in particular those who are and have been 
traditionally underserved. Toward that end, our mission is to advance student-centered learning for all 
students, by supporting teachers designing and leading schools, and by advocating for policy that is open to 
innovation. Read more at www.educationevolving.org. 


We are immensely grateful to the more than 50 students, families, educators, policymakers, community 


advocates, researchers, and business owners we spoke to for this paper. We do not list them here since we 
asked them to speak candidly from their experience, and indicated we would not identify them by name, but 
each contributed in unique ways to the final product. Thank you! 
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As goes the adage, what gets measured gets done. This statement has proven true, over time and across 
sectors! In education, the outcomes set for students and schools, and the measures used for those 
outcomes, influence whether learning can truly be designed with students at the center 


In recent years, there has been an unprecedented focus on measurement and data in public education, 
beginning most notably with the 2002 passage of the No Child Left Behind Act (NCLB) and continuing 
through the Every Student Succeeds Act of 2015 (ESSA). To be sure, these policies have made important 
contributions. They brought attention to egregious and unacceptable inequities among students in very 
important areas of academic achievement,? and some analyses have shown modest student gains in 
tested subjects over this period.* 


At the same time, these policies have had unintended 
consequences. The emphasis on assessments of reading and 
math has corresponded with a narrowing of curriculum to 
focus on these subjects,° and data has come to be associated 


more with accountability than with improving teaching define and measure student 
and learning. Perhaps most importantly, the measures and school outco S 
emphasized in this era do not capture a full picture of what 

matters to equitably prepare all students for success in 

college, career, civic participation, and life® 


To be clear, we are not advocating to eliminate standardized tests. Rather, we assert it's time to expand 
how we define and measure student and school outcomes to also include broader and deeper sets of 
knowledge and skills that are valued by students, families, and communities,’ and are necessary for 
students’ success in the 21st century. And, it's time for assessment and data to be seen, first and foremost, 
as tools to inform student learning and school improvement, in addition to the purpose of accountability. 


Fortunately, a growing movement of educators and policymakers around the country are doing just that. 
These pioneers are defining and measuring broader, deeper, and more student-centered outcomes for 
students and schools. The purpose of this paper is to synthesize the research and ideas that underlie 
this movement, in order to grow a shared understanding, set of terminology, and vision for the future. 


Toward that end, this paper is divided into three parts: 


¢ Part 1 describes the student and school outcomes that research has shown to be important for 
student success in life, and that are valued by students, families, and communities. 


e Part 2 explores the various purposes for measurement in public education, and some of the specific 
measurement strategies that are used to capture the outcomes described in Part 1. 


e Part 3 draws on the analysis from the first two parts to present a vision and specific ideas for 
educators and policymakers seeking to define and measure student-centered outcomes. While our 
vision is focused on Minnesota, where our organization is based, the ideas presented in this paper are 
also relevant to other states. 
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me to expand how we 


Defining Outcomes 


In this part, we present our findings on the student and school outcomes that 
research has shown to be important for student success in college, career, civic 
participation, and life—and that are valued by students, families, communities, 
educators, and employers. Our research consisted of a comprehensive literature 
search, an examination of 15 existing outcomes frameworks, and interviews with over 
50 students, families, educators, policymakers, community advocates, researchers, 
and business owners (see Appendix A for research methodology). 


Our intention in this part is not to define the full set of outcomes that all states, 
districts, and schools must use, but rather to present a framework or “menu” of 
ig=s-{-t- |e pte OY- (el, 4-16 Mo] U i mee) aa(= Mele) aar-liat-m-]alemee) pn) e\-14=1ae( soma ar-lemee)palpalelalia(-1-mer-lakele-\iVme)p 
as they decide what student-centered outcomes they will define and measure. This 
ViVi Um o=me(=t-e] a] oY-\o aU] aual-lallamual=mualiqem-lacemulat-l@ ey-)ame) meal (om oy-/ol-) 0 


Note that in this paper we use the word “outcome” in a broad sense, to describe both 
student learning outcomes (for example, math knowledge, critical thinking skills, and 
developing a positive sense of identity) as well as other student and school outcomes 
that support learning (for example, engagement, safety, and school culture). We now 
consider both of these broad categories in turn. 


, districts, and schools must use an 
inclusive, asset-oriented process to identify 
the specific student-centered competencies 


they will define and measure, to be 
responsive to the students, families, and 
communities they 
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Student Learning Outcomes: 
Four Key Competency Domains 


We concluded from our research that, to be prepared for success in the 21st 
century, students need to develop competencies in four primary domains, as 
shown in the following figure. While we have chosen our own names for these 
domains, we do not claim originality; these domains are emerging as a consensus 
across the learning science research and the various outcomes frameworks we 
reviewed. 


A Call for Inclusive Definitions of Competencies 


In this paper, we define a 
competency as a learning 
(olU]efolant=nelm ala mant-sie-)aaer-ia 
be demonstrated. The three 
main types of competencies 
include: 


¢ knowledge, for example, 
an understanding of the 
TaxelUisidat-l@nconvce) (Old lelan 


skills, for example, writing 
persuasive arguments, and; 


mindsets, for example, 
a belief in one's own 
capabilities. 


All three types of competencies 
are included in each of the four 
domains shown in Figure 1, as 
will be described further in this 
section. 


Figure 1. Research shows that developing competencies in 
four key domains is important for student success in college, 
career, civic participation, and life in the 21st century. 


In conducting our research, we were mindful that different communities have distinct ideas about 
outcomes for learning, and so sought to speak with and read the work of individuals representing a variety 
of cultural, racial, ethnic, linguistic, geographic, socioeconomic, and ideological backgrounds and identities. 


We found the four domains described in this section are generally valued across different communities. 
At the same time there were important differences, and we assert that states, districts, and schools must 
use an inclusive, asset-oriented process to identify the specific student-centered competencies they will 
define and measure, to be responsive to the students, families, and communities they serve.® This will be 


described further in Part 3: Moving Forward. 
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Additionally, the idea of “developing competencies” must also be inclusive. Many students exhibit mastery 
of these competencies in their lives outside of school, yet in school many of those same students are 
disengaged, reporting that education is not relevant to their lives nor reflective of their histories and 
identities.? As such, we contend that educators must work with students to create supportive environments 
where allstudents see learning as relevant, and so want to express, expand, and demonstrate—in academic 
contexts—the competencies they often already possess. 


We now turn to consider each of these four domains, including the key competencies within each domain, 
the primary sources of evidence that make the case for that domain, and the “blurry lines” that exist 
between that domain and other domains. 


Domain 1: Content Competencies 


Also called: subject-area knowledge; academic content; foundational knowledge. 


Overview: This domain is fairly well represented in the state standards in Minnesota and other states, 
however some important areas described below—such as self, cultural, technology, and media 
understanding—are not as explicitly addressed. While the competencies in this domain are primarily 
knowledge, some domain-specific skills are also included; for example, mathematical problem solving. 


Key competencies in this domain: 


e Academic disciplines. Understanding of language arts, mathematics, science, and social studies— 
and including knowledge traditions from a variety of cultures and backgrounds. 


e Self, cultural, and global knowledge. Understanding one’s personal history and that of one's 
ancestors, cultures that are different from one’s own, macro-level global trends, and world languages. 


¢ Technology and media literacy. Knowing how to navigate vast amounts of information available 
on the internet, evaluate the credibility of sources, and use technology to create and disseminate 
information. 


Evidence: The evidence for this domain comes primarily from the cognitive science principle that all higher 
order mental processes build on foundational content knowledge and academic abilities, in particular on 
numeracy and literacy." Assessments of this domain—namely, standardized tests in reading and math— 
show correlation with success in other areas of life’? At the same time, some point out that high test 
scores are the results of other skills co-occurring with content knowledge and academic ability,’ such as 
persistence, study skills, and short term memory, and are correlated with other characteristics such as 
family socioeconomic status and access to supplementary academic resources. 


Blurry lines: As students exercise higher order critical thinking and problem solving within the Content 
Competencies domain—as is common, especially in middle and high school—they use a number of skills 
that fall more within the Cognitive Competencies domain, which we describe next. 
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GES: Domain 2: Cognitive Competencies 


Alsocalled: cognitive skills; higher order thinking skills; cognitive strategies; fluidintelligence; 
21st century skills i.e. "the Four Cs". 


Overview: This domain includes higher order thinking skills that are applied within and across disciplines 
to solve problems, generate ideas, analyze and synthesize information, formulate arguments, and create 
plans for action. Recent iterations of the state standards in Minnesota and other states incorporate more 
of this domain, although even more could be done.* 


Key competencies in this domain: 


¢ Critical thinking. Conceptualizing, analyzing, synthesizing, and evaluating information gathered 
through observation, experience, reflection, and reasoning—and using that information to guide 
belief and action. 


¢ Creativity (also called inventiveness or innovative mindset). Using imagination, original ideas, or out 
of the box thinking to accomplish tasks, solve problems, produce works of art, or create inventions.’ 


¢ Communication. Possessing skills in writing, speaking, listening, and other nonverbal dimensions of 
communication. 


Evidence: Much of the evidence for this domain comes from literature on the importance of overall 
cognitive capacity or “general intelligence”, which has been shown to overlap with and be part of all of the 
competencies in this domain.” Proponents of this domain point to the overall malleability of intelligence,’® 
and the fact that encouraging positive mindsets can improve competency in this domain.’? Research has 
long shown that competencies in this domain are highly context-dependent (i.e. do not “transfer” very well 
across fields*°), which underscores the importance of designing learning experiences that develop both 
Cognitive Competencies and Content Competencies concurrently.”' Finally, employers usually rank the 
skills in this domain high on their list, indicating they find them to be extremely important to employees’ 
success on the job.’ 


Blurry lines: Researchers differ in how they categorize interpersonal skills, especially communication 
and collaboration. Some frameworks (such as the influential 21st Century Skills Framework) group these 
into Cognitive Competencies, whereas others group them into Social-Emotional Competencies, which we 
describe next. 
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Domain 3: Social-Emotional Competencies 


Also called: character; noncognitive skills; interpersonal & intrapersonal skills; soft skills; 


habits of success. 


Overview: This domain includes competencies related to one’s understanding of and relationship with 
self and with others. The ideas that underlie this domain appear in writings on youth development from 
a variety of traditions and cultural heritages.”? In the world of academic research, the initial work on this 
domain was done mostly in the field of personality psychology.“ While the list below primarily uses terms 
that appear in academic research, we acknowledge that different knowledge traditions use different 
terms for these competencies. 


Key competencies in this domain: 


Self-efficacy (also called agency). Having conviction and confidence in one’s ability to be successful 


with a situation, task, activity, or pursuit. 


Positive self-image (also called hope, positive future self, 
positive identity). Having a positive mental image of who 
one is, including the various dimensions of one’s identity, 
and what lies ahead in one’s future. 


Growth mindset. The belief that intelligence and ability 
are malleable, and that one can improve over time with 
effort. 


Curiosity (also called intrinsic motivation). Having a natu- 
ral desire to learn more about and explore the world, and 
the experiences of others. 


Self-awareness (also called mindfulness). The ability to 
recognize one’s emotions, thoughts, values, and behav- 
iors—and how they relate to one another. 


Self-management (also called self-regulation, self- 
discipline). The ability to regulate one’s emotions, 
thoughts, and behaviors in different situations. 


Perseverance (also called conscientiousness, resilience, 
grit, persistence). The ability to stay on task in working 
towards a solution or goal, in spite of setbacks. 


CG. in this domain should 


emphasize understanding— 
on the part of all students, 
including those who identify 
with the dominant culture— 
how people from different 
cultures and backgrounds 
might define and experience 


these competencies ~ SF 


Social awareness (also called empathy, respecting others). The ability and desire to understand the 
experience of others, especially those from cultures and perspectives different from one’s own. 


Relationship skills (also called social skills). The ability to build and maintain healthy and rewarding 


relationships with others. 


Collaboration (also called teamwork). The ability to work together productively with others in pursuit 


of acommon goal or endeavor. 


Generosity (also called prosocial orientation, civic responsibility, ethical behavior, spirit of service). 
A belief that all people matter, and a commitment to service and stewardship of one’s community, 


environment, and the world. 
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Domain 3: Social-Emotional Competencies (continued) 


Care must be taken not to define these competencies only in reference to Eurocentric, dominant cultural 
norms.” Rather, learning in this domain should emphasize understanding—on the part of all students, 
including those who identify with the dominant culture—how people from different cultures and 
backgrounds might define and experience these competencies differently. As an example, mastering 
the competency “relationship skills” would include understanding how maintaining eye contact in 
conversation can be seen as polite in some cultures, while other cultures consider averting one’s gaze to 
be a sign of respect. 


Evidence: Most of the initial research on these competencies was conducted in the early childhood, youth 
development, and out of school spaces,”° although research in the K-12 space has expanded significantly in 
recent years. While there isn’t space in this paper to review the evidence for each individual item in the list 
on the previous page, there is conclusive evidence that competencies in this domain are malleable rather 
than fixed,?”? can be intentionally developed through learning experiences and interventions,”® support 
learning in other domains,”? and are correlated with success in one or more areas of college, career, civic 
participation, and life.*° See Appendix A for more information and a full bibliography. 


iD Domain 4: Navigational Competencies 


de 


Also called: college and career skills; practical skills; wayfinding skills; transition knowledge 
and skills; sense of purpose. 


Overview: This domain includes practical competencies a student needs to navigate the quickly changing 
world of work and life after public education. Of all four domains, this is the newest domain to be commonly 
included in frameworks, with less common language developed and less consensus as to what it includes. 


Key competencies in this domain: 


e Exploration and planning skills. Identifying interests, exploring potential careers and career 
pathways, and setting longer-term life goals. 


e Opportunity seeking skills. Applying to college and for financial aid, seeking scholarships, searching 
for jobs, and doing job interviews. 


e Social capital skills. Cultivating networks, seeking mentors, and drawing on one’s social capital to 
identify opportunities for professional advancement. 


e Practical life skills. Financial literacy and managing money, finding a place to live, preparing healthy 
food, and caring for one’s physical and mental health. 


Evidence: As the frequency with which people change jobs and careers has increased—driven by factors 
such as globalization and technological automation—so too have calls for more emphasis on this domain 
in school?! Most evidence for this domain comes from evaluations of programs that successfully help 
students develop skills in this domain,?? as well as from studies showing how significant barriers such as 
complicated admissions and financial aid forms are to some students and families.*? Evidence for the social 
capital dimensions of this domain comes from sociology, where the presence of such capital is identified 
as a primarily factor influencing economic mobility.** 
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Other Outcomes that Support Learning 


At the beginning of Part 1, we defined “outcomes” to include both student learning outcomes, as well as 
other student and school outcomes that support learning.*® In this section we explore the latter category. 


We acknowledge that in practice the lines between learning outcomes and outcomes that support 
learning can be blurry. For example, while “sense of belonging” appears below as an outcome that supports 
learning, arguments could be made it is also a learning outcome in and of itself. We accept this definitional 
ambiguity but contest that, either way, the outcomes described in this section are important. 


Other Student Outcomes that Support Learning 


Some of the student outcomes that have strong evidence in research, and per interviews and surveys also 
emerged as important to students and families, include: 


e Engagement. The student sees school as relevant to their lives and expresses interest in learning. 
They are regularly engaged in deep, active, rigorous learning that involves retrieval, synthesis, and 
processing.?” 


e Relationships and Expectations. Adults and peers at school trust and respect the student, believe 
in their potential as a person and in their abilities as a scholar, and hold them to high expectations.?® 


e Sense of Belonging. The student feels valued and embraced by the school community, and the 
broader community of which it is a part. They feel they can bring all dimensions of their identity with 
them to school without fear, shame, or embarrassment.? 


Other School Outcomes that Support Learning 


Some of the school outcomes that have strong evidence in research, and per interviews and surveys also 
emerged as important to students and families, are listed below.*° Unless otherwise noted, the evidence 
for many of the school outcomes below draw on the extensive literature on school culture and climate.” 


e Safety. Members of the school community (educators, students, and families) treat each other with 
respect and feel safe from physical and psychological harm.” There are rarely or never incidents of 
bullying or violence, and restorative practices are used where possible rather than punitive disciplinary 
actions. 


e Culturally affirming environments. Students’ cultures and identities are respected, honored, and 
celebrated in school. Adults and students make an effort to understand each other’s cultures and 
traditions. Students have adult role models in the school with whom they share elements of cultural, 
racial, and ethnic identity.” 


e Community involvement. Families and community members feel welcome at school, have 
opportunities to participate at times and in ways that work for them, and are included in school 
decisions.** The school has strong relationships with local organizations, businesses, and other 
community partners, and draws on those partnerships to provide real-world learning opportunities 
for students. 


e Strong and stable staffing. The school is staffed by highly competent, committed, and culturally 
conscious educators.** These adults like working at the school—with minimal turnover from year to 
year—and have the opportunity to access and shape professional development opportunities. 


e Educational opportunities. The school offers a rich and rigorous set of program offerings, either 
through courses/classes, or non-traditional avenues such as projects or internships. Students are able 
to participate in AP, IB, CTE, PSEO and college in the schools courses, as well as music, arts, athletics, 
internships, and other electives. 
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e Strong finances. The school spends its funds in accordance with an approved budget that is well- 
aligned with its vision and mission, and has protocols to prevent waste and fraud. The school receives 
equitable resources to meet the needs of the students it serves (noting that this is often not a factor 
the school can control). 


e Strong operations. The school has strong, functioning systems in place with respect to human 
resources, transportation, facilities, enrollment, and other back office systems that support teaching 
and learning. 


¢ Strong governance. The school’s board or site council upholds their governance duties, ensuring that 
all students learn and that the school operates legally and ethically. The board operates in positive 
partnership with teachers and administrators, and avoids micromanaging or becoming directly 
involved in school program and personnel decisions. 


Here in Part 7 we have described the student and school outcomes important for student success 
in the 21st century. We acknowledge an obvious next question emerges: How can these student- 
centered outcomes for schools and students be fostered? 


That is a substantial question, which is beyond the scope of this paper, but which Education Evolving’s 
Theory of Change, and the rest of our organization's work, addresses directly.** In sum, we assert that 
student-centered learning experiences” —enabled by larger professional roles for teachers, and state 
and district policies which create openings for innovation—are the route to these broader, deeper, 
more student-centered outcomes. Read more at: www.educationevolving.org/theory 


Student-Centered Lead To Student-Centered 
Learning Experiences Outcomes 


(the topic at the core of other (the topic of this paper) 
work by Education Evolving) 
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Measuring Outcomes 


While there has been some movement over the last decade to define and value the 
broader, deeper, more student-centered outcomes described in Part 1, measurement 
of those outcomes is still lagging. Partly, this is fine; not everything needs to be 
measured, and simply setting new goal posts can help students, teachers, schools, 
districts, and states to reorient toward more student-centered outcomes. 


At the same time, we assert it is critically important to increase measurement of 
student-centered outcomes. In this part, we describe why measurement and data are 
useful and important, explain three main purposes for which they can be used, and 
finally inventory nine main strategies that we found are currently used to measure 
student-centered outcomes. 


Our intention in this part is to present a framework of possible measurement purposes 
and strategies for state, district, and school communities to consult as they craft their 
own cohesive approach to measuring outcomes. This will be described further in the 
third and final part of this paper. 
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Why Measurement? Three Main Purposes 


Why are measurement, assessment, and data useful and important for advancing student-centered 
outcomes? Practically speaking, there are three main purposes—each held by different stakeholders 
within public education—which together answer that question. We consider these three purposes in turn. 


1. To Inform and Improve Teaching and Learning 


First, and arguably most importantly, students and educators working directly with students can use data 
to take a pulse on where students are at and adjust teaching and learning accordingly. This can include 
discerning areas where individual students are struggling in order to identify needed supports and 
interventions, or understanding where a class or group of students is as a whole, to know what concepts 
need review. Data can also help students to take 


ownership of their learning, celebrate their growth 
and successes, and accelerate boldly ahead toward 
the goals they set for themselves. . 
st, and arguably most importantly, 


Additionally, administrators, teachers, and other 


educators at the school or district level can use students and educators working 
data to illuminate which programs and strategies directly with students can use data 
are working when, where, and for whom, and 

identify when adjustments might be needed. totakea pulse on where students 
Writings about data for this purpose use terms are at and adjust teaching and 


learning communities (PLCs)’® or “networked 
improvement communities”.*? 


like “formative assessment” or “assessment for [ 2 di [ 
learning”, which often happens in “professional earning accordingly. y y | 


2. For Research and Evaluation 


Researchers use data to produce generalizable conclusions that can inform practice at scale. Rather than 
informing practitioners’ work with specific students or schools, the focus of research is to understand 
broadly what learning models, interventions, and instructional strategies work well, and under what 
conditions. Of all three purposes, research is furthest along in measuring student-centered outcomes; 
indeed, many of the measures now being contemplated for other purposes were originally created for 
research. 


3. For Transparency and Accountability 


With public education provided for in state constitutions and funded with tax dollars, policymakers and 
the public value using data to understand how students, schools, and districts are doing.°° Namely, families 
report that they value using data—for example, data on a school’s “report card” as required under federal 
law—to help select aschool to attend, to support their schoolin areas where it is struggling, and to advocate 
for their students.*' This purpose is often called “data for transparency” or “soft accountability”. 


Additionally, policymakers, school districts, and charter authorizers charged with overseeing public schools 
benefit from using data to monitor how individual schools are doing, in order to celebrate success and 
provide targeted support, interventions, and (although rare, at least in Minnesota) closure. This purpose is 
typically called “data for accountability”. 
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A Common Theme: Promoting Clarity and Objectivity 


Across these three purposes, a common theme is the role measurement and data can play in promoting 
objective, comparable conclusions across different students and schools. While hardly perfect, focusing 
on reliable and valid measurement can help combat bias and 


promote clear and high expectations for all students.°? 
Next, we describe specific measurement strategies that can be 
used for these three purposes. 
ocusing on reliable and valid 


Nine Measurement 
S iesC ly Used measurement can help combat 
mateo eS ~onuony Uae bias and promote clear and high 


From our research, we identified nine strategies commonly . 

used to measure the student-centered outcomes described expectations for all stu 
in Part 71. While most of these measurement strategies are 

focused on the individual student—which yield data that could 


be aggregated to the school, district, or state level—the ninth 
and final strategy is focused at the school level. 


1. Achievement tests are a classic and long-standing measure of student learning outcomes. Most of the 
time they use a multiple-choice, true/false, or short answer format. Tests requiring longer, more involved 
answers begin to blur the line into performance assessments, which are described below. 


Examples: The Minnesota Comprehensive Assessments, a series of multiple choice tests given 
to Minnesota students in English language arts, math, and science. Or, the Northwest Evaluation 
Association Measures of Academic Progress assessment (NWEA MAP), which is commonly used 
for formative purposes. Additionally, educators often create achievement tests at the classroom 
and school level, for example multiple-choice formative quizzes or end of course exams. 


2. Embedded assessments are not standalone events, but rather use data collected as students participate 
in learning activities. For example, a student who plays an online computer game to practice Algebra 
generates data on concepts they do and do not understand; as a more futuristic example, researchers are 
developing facial expression recognition technology to monitor a student as they participate in a virtual 
reality world in order to assess their “curiosity.” With student learning increasingly assisted by technology, 
the amount of data that could be examined as embedded assessments, without interrupting student 
learning, is growing rapidly. 


Example: In the popular software program DreamBox, students engage in math games and 
activities to practice their skills. At the same time, the software tracks their level of mastery of 
various competencies, which teachers can review and offer additional help to students as needed. 


3. Performance assessments ask a student to solve a complex problem, demonstrate their work, and/ 
or produce a tangible final product. Their results are then evaluated against a rubric, a model standard of 
work, or both. Many dimensions of performance can be considered in a single assessment; for example, 
a reviewer could look for evidence of mathematical knowledge, writing, creativity, and teamwork. While 
performance assessments have been around for decades, in recent years advances in both technology and 
psychometric practice have increased their feasibility, reliability, and validity. 


Example: New Hampshire’s Performance Assessment of Competency Education (PACE) pilot 
gives performance assessments to students grades 3 through high school. As an example, one 
PACE high school math task might be: “Your town’s population is predicted to increase over the 
next 3 years. Please analyze, write, and present to the city council a proposal to build a water 
tower that can hold 40,000 cubic feet of water.” 
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Nine Measurement Strategies 


Commonly Used (continued) 


4. Portfolio assessments (also called work sampling) involve evaluating student performance based on 
a sample of their work selected by the student themselves, by their teacher, or randomly. Evaluators use 
techniques similar to those used for performance assessments—including rubrics and/or model samples 
of work. An evaluator may be the students’ teacher, someone else at the school, or an external party. 


Example: Students at Mission Hill School in Boston prepare and defend portfolios of their work 
at the end of seventh and eighth grade. Students select pieces of work to demonstrate their 
achievement and growth in literacy, mathematics, science, theme studies, and art. Students also 
demonstrate evidence of developing the school’s five “Habits of Mind” and four “Habits of Work”.*4 


5. Field observations are conducted by educators present while a student is engaged in learning. These 
assessments have long been used in early childhood education; for example, a teacher may count the 
number of times a child shares their toys with others. For older students, teachers may record perceptions 
such as how engaged or on-task a student is. Teachers often calibrate their behavioral observations in 
reference to arubric. 


Example: Montessori educators have developed a number of resources, including forms to tally 
frequencies of certain behaviors, for tracking the progress of young children on skills such as 
concentration, independent agency, and cooperation. 


6. Surveys are typically used to assess either how people feel (i.e. whether their school’s climate feels 
safe, respectful, supportive, etc.) or how they perceive their abilities (i.e. knowledge, skills, mindsets, 
and behaviors). They can be given to a variety of audiences, including students, families, teachers, and 
administrators. Often times a survey will ask these different groups a similar set of questions in order 
to get multiple perspectives on what is being measured. For example, a paired survey that measures the 
Social-Emotional Competencies described in Part 1 might ask a student to self-report on their perceived 
level of mastery of acompetency, and also ask their teacher to report on perceptions of that same student. 


Example: The 5Essentials survey, used by over 6,000 schools and districts across the country, 
including several large districts in Minnesota as well as all schools and districts in Illinois (where it 
was developed), asks students, teachers, administrators, and families a series of questions about 
five school characteristics found to be important for school success.*° 
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7. Longitudinal data is collected on students throughout their lives, including and especially after they 
graduate from high school. The most common measures include: college enrollment, remediation, 
persistence, and graduation; as well as workforce participation and earnings. Less common measures 
include: civic participation (such as voting), health outcomes, and more. The logistical and ethical challenge 
with these measures is that they require accessing sensitive information from various databases. The 
upside is they help tell the “true” story of whether students have been prepared for success in their lives 
beyond school. 


Example: The Minnesota State Longitudinal Education Data System (SLEDS) tracks data on 
individual students from kindergarten through post-secondary education and into the workforce, 
including college enrollment, retention, and more. 


8. Administrative data is information collected about the programs, activities, and incidents that take 
place as part of a school’s learning program. Common measures include: attendance and tardiness rates, 
suspension and expulsion rates, credit attainment, internship placements, and participation rates in AP, IB, 
CTE, PSEO, or college in the schools courses. Additional measures at the school (rather than student) level 
include: teacher retention rates and student waiting list size. 


Example: A number of state and district data systems calculate an “On-Track Indicator,” which is 
an overall metric of whether a student is on track to graduate.°° The specifics of the metric vary 
by state, but they commonly measure the percentage of ninth grade students who earn enough 
credit to be one-fourth of the way to meeting high school graduation requirements. 


9. External reviews are conducted by someone external to the school who visits the school with the 
purpose of observing student learning, school culture, or other dimensions of school performance, 
usually aided by tools such as checklists and rubrics. Often times this person is either with the entity 
overseeing the school (such as a school district or charter authorizer), or a reviewer they have contracted. 
While sometimes these reviewers make general judgments about whether a school is helping to foster 
competencies in students, this strategy is different from the others in that it is explicitly targeted at the 
school level rather than individual students. 


Example: In the Annual School Review Process, used in the United Kingdom, trained reviewers 
visit schools and evaluate their operations and program against a set of criteria. A second example 
common in Minnesota is the regular reviews and site visits that charter school authorizers do of 
the schools they authorize.°” 
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An Authentic, Integrated, Multi-measure 
System 


A fair question readers likely have at this point in the paper is: 


How can teachers, schools, districts, and states that want to expand the 
student-centered outcomes they measure, do so within the constraints 
of limited time and energy, and without taking time away from learning? 


In a worst-case scenario, one can imagine a separate measurement event each hour of the day: first a 
diagnostic reading assessment, then a critical thinking performance task, then a curiosity skill survey, etc. 
With students, teachers, and the public already experiencing high levels of assessment fatigue,°® it would 
make anyone’s head spin. Fortunately, there is an alternative: 


moving toward an authentic, integrated, multi-measure system. 

This shift requires at least three significant changes in mindset. 

First, we must see measurement primarily as an exercise tunately there is an 
b] 


in “collecting authentic evidence”, often work produced by 


students as part of the learning experience. For example, the alternative: moving toward 


performance, portfolio, embedded, and observation-based 


assessments described earlier in this section can be seamlessly an authentic, integrated, 
(and sometimes invisibly) integrated into learning, without 7 

requiring a separate “assessment” or “test day”. multi measure sy 

Second, we must use measurement strategies that look at 

several outcomes at once. For example, many schools using 

project-based learning require end-of-year student projects that involve research, writing, and presenting 

on a multi-dimensional topic. A complex piece of work like this provides students the opportunity to 


exhibit many different competencies, each of which can be reliably evaluated using the measurement 
strategies above. 


Third, we must accept that multiple measurement strategies used together can paint a more holistic, 
integrated picture of outcomes.* For example, a student’s “persistence” might be measured both by a 
survey of the student and their teacher, as well as by a performance assessment that requires persistent, 
extended inquiry on a complex topic. 


So how, specifically, do we move forward? Part 3 seeks to answer that question. 
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Table 1. Choosing measurement strategies for student-centered outcomes 


The following table contains our estimations of the general appropriateness of the measurement 
strategies described in Part 2, for each of the outcome domains described in Part 1. 


This table is intended to help educators, schools, districts, and states get ideas for assembling their own 
integrated measurement system. We compiled this table based on general findings from interviews 
and research, however we emphasize that the true appropriateness of a measurement strategy always 
depends on the combination of the purpose of the measure (i.e. formative, accountability, etc.), the specific 
measurement instrument used, and the specific outcome or competency measured. 


KEY 
Not commonly used to measure this outcome domain 


Sometimes used for this outcome domain, 
or used for a portion of the domain 


Commonly used for this outcome domain 


ee What are you seeking to measure? 
Other Outcomes 


Pees Content Cognitive OS Navigational es Srnpe 
measurement Emotional Learning 


strategies to use oe eee ee cne Competencies SLURS (school culture, 
engagement, etc.) 


Embedded 
Assessments 
Performance 
Assessments 
Portfolios/Work 
Sampling* 
Administrative 
Data 


* Indicates special care must be taken with respect to reliability, validity, and overall appropriateness 
of this measurement strategy, if it is used for the purpose of accountability. 
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Moving Forward 


Parts 1 and 2of this paper yielded the conclusion that equitably preparing all students 
for success in the 21st century will require defining and measuring a broader, deeper, 
more student-centered set of outcomes for both students and schools. But what does 
that look like in practice, and how do we get there? 


In this final part, we draw on the research and analysis from Parts 1 and 2to presenta 
high level visionfor anewreality at three distinct levels—for the state of Minnesota; for 
school and district learning communities; and for individual students and families— 
and propose ideas for next steps that pioneering educators and policymakers should 
consider. 


Importantly, this vision and these ideas are not intended to be prescriptive, but rather 
to spark aninclusive conversation about next steps. Nor do we claim this vision as ours; 
rather, many of the leaders we interviewed in writing this paper are living portions of 
this vision already, and inspired what we write here. We present this vision and these 
ideas with humility and respect for those already blazing trails. 
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Three Levels Inform a Full Set of Outcomes 


In Minnesota, there are three main levels at which outcomes 
can be defined and measured—the state, the learning 
community (ie. the school or district), and the individual 
student and their family. Together, these shape the complete 
picture of target outcomes for a student, with each level 
including and expanding on the preceding level, as illustrated 
in Figure 2. As we describe below, this is both the way things 
are because of state law but also as it should be. 


State. Minnesota law requires state standards for what 
students should learn in language arts, math, science, social 
studies, and physical education.® In general, having state 
standards is important because public education exists in 
the state constitution and is funded with public dollars® 
and because some learning outcomes, such as foundational 
literacy, are so critical to student success that all students 
can and should master them.” With that said, we do present 
some ideas for improving what's included in state standards, 
below. 


Learning community. Under Minnesota law, schools and 
districts are free both to decide how, concretely speaking, the 
standards will be covered for their local learning community, 
and to define additional learning outcomes beyond those 
specified in state standards. This is as it should be; each 
community is unique in its composition and what it values 
for students to learn. 


Individual student and family. Under Minnesota Law, defining 
individual student-level learning outcomes is fully permitted 
and, in fact, under the 2013 World’s Best Workforce law, 
districts are required to create personal learning plans for 
students in which they “explore their educational, college, 
and career interests, aptitudes, and aspirations.”© This is as it 
should be; every student and their family have unique values, 
hopes, and dreams for learning, which should be considered 
by schools and teachers.®* 


Figure 2. Outcomes defined at three 
levels together form a complete set of 
target outcomes for a given student. 


hool and district learning 
communities need to lead the 
movement toward defining 
and measuring more student- 
centered outcome 


While the following section presents ideas for defining and measuring student-centered outcomes at all 
three of these levels, we emphasize that school and district learning communities (the “middle level” 
in the list above) need to lead the movement toward defining and measuring more student-centered 
outcomes. Local learning communities are where outcomes and assessments become “real”, by shaping 
and informing the actual day-to-day learning experiences students have. Our ideas for states below should 
been seen as supporting a movement led by educators and communities. 
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Specific Ideas for Moving Forward 
A. Ideas for States 


fat) Revise state standards to include more student-centered knowledge and skills 


While having state standards is important for the reasons described above, imagine if future revisions of 
those standards better reflected the student-centered outcomes described in Part 1. Namely, imagine if 
standards emphasized Content Competencies around media and technology literacy, global and cultural 
understanding, and personal history and identity exploration. And, imagine if standards prioritized 
Cognitive Competencies such as critical thinking and problem solving, over superficial coverage of a wide 
base of content.® 


This approach to standards is used by the highest performing, most educationally equitable countries 
in the world, including Singapore, Finland, and Canada®’” —and increasingly used in the United States. 
It emphasizes what researchers call “focus and coherence,”® by prioritizing deep, enduring concepts and 
foundational habits of mind, which are applicable to future academic content and to students’ future 
lives.” 


@ Make state “report cards” easier to use, and add a few new student-centered measures 


Federal law requires Minnesota to create a “report card” website that provides information on all of 
the state’s public schools and districts.” However, many of the people we interviewed for this paper 
commented that the current website was difficult to use. Imagine if the report card website was more 
useful to students and families in making choices about which school to attend, and to educators and 
communities in helping support improvements at their school. Namely, imagine if it was easier to navigate 
and understand, and more accessible to families whose native language is not English. 


And, imagine if the report cards featured a limited number of new indicators that painted a more holistic 
picture of student and school outcomes.” To be clear, the idea is not that statewide measures capture a 
comprehensive set of student-centered outcomes; as described previously, we believe the local learning 
community is the level at which outcomes and assessments can best shape and inform the day-to-day 
learning experiences students have. However, local communities do not exist in isolation, and our end 
goal is statewide change toward student-centered 


outcomes. In order to signal that the state values 

student-centered outcomes, to provide transparent 

information to families and policymakers, and to . : F 
pie ell a data narrative that will drive 


tell a data narrative that will drive the movement 
in the long term, we need a small set of uniform, the movement in the long term, 
comparable, statewide measures of student- . 

centered outcomes beyond the conventional we need a small set of uniform, 


academic measures that predominate today. comparable, statewide measures of 
A few possible such measures—some of which student-centered out 


MDE already has or will be rolling out—include: 
consistent attendance; student — disciplinary 
incidents; teacher experience levels and retention 
rates; credit attainment and on-track progress toward graduation; participation in AP, IB, CTE, PSEO, 
college in the schools, internships, or other advanced learning opportunities; achievement and graduation 
rates that are adjusted for the amount of time a student has been enrolled at the school; and post- 
secondary enrollment and workforce participation rates after graduation. Most of these indicators would 
not require schools and districts to collect new information, although some updates to state data systems 
would be necessary. 
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© Implement a yearly, statewide survey of students, families, and educators 


Another indicator commonly included on report cards, appearing in at least 11 other states, are the 
results of statewide surveys of students, families, and/or educators.” These surveys can yield important 
information about both the development of Social-Emotional Competencies in students, as well culture, 
climate, and student engagement in schools. 


Imagine if Minnesota revamped its current statewide student survey to better fulfill this purpose. Namely, 
imagine if it was given every year rather than every three; was taken by students, families, and educators 
rather than only students; and measured a set of student and school outcomes consistent with Part 1 of 
this paper. A diverse and representative group of educators, researchers, and community members should 
be involved in this revamp process, to ensure the survey is inclusive of those from different cultures and 
backgrounds, and useful to educators, students, and families. 


@ Improve the state’s standardized tests 


While adding a few new statewide indicators of student-centered outcomes is important, Content 
Competencies still matter, and standardized tests have an important role to play in the state's overall 
measurement system. However, imagine if state tests were less intrusive for students, and more helpful 
for educators and families wanting to support them.” 


For example, imagine if the state test could be taken in separate parts throughout the school year rather 
than all at once at year end (as is possible under ESSA), with results available the next day.’° Or, imagine 
the state partnering with a company that produces formative assessments (like the NWEA MAP) so that 
the same tests currently used for formative purposes could also serve for accountability with, of course, 
important modifications and accommodations.’”° Or—even bolder still—imagine giving districts the option 
to embed standardized state questions into end of course exams, with safeguards in place to ensure 
question security. 


© Launch a state performance assessment pilot, perhaps starting with science 


Imagine if the statewide assessment system better measured 

Cognitive and Social-Emotional Competencies through the 

use of performance assessments.” As described in Part 2, in : 4 s 
performance assessments students are given a complex task sl ne if the statewide 


involving higher order thinking. Their work is then evaluated assessment system better 
by teachers (with safeguards to ensure reliability and 

consistent high expectations for all kids), and used for formal measu red Cogn itive 

state proficiency determinations.”® Students would still take ‘ F 
standardized tests in some years; in other years they would and Social-Emotional 
take these performance assessments. Competencies th ro ugh 
Minnesota has an opportunity to move in this direction over the use of pe rform 

the next two years. The state science standards are currently 

up for revision, and when that process is complete the state assessments. 


science test will need to be revised. Imagine if MDE piloted a 

performance assessment in science with a subset of schools 

and districts, much like the state of New Hampshire did (see case study below). Depending on the results, 
the pilot could be scaled to other schools and districts, and to math and reading. 
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© Explicitly authorize competency-based education in Minnesota 


Competency-based education is an approach in which student learning is oriented around mastering 
clearly-articulated, measurable learning outcomes (i.e. competencies). Students have choices in how they 
meet those competencies, receive differentiated support based on their individual learning needs, and 
advance by demonstrating mastery of competencies (rather than time in seats). There are some schools 
and districts that are already implementing competency-based education practices in Minnesota, but the 
majority of educators we spoke with indicated they do not read current state statute as permitting this 
form of education. 


Imagine if Minnesota law explicitly authorized competency-based approaches to defining and measuring 
state standards and local learning outcomes. Education Evolving worked with legislators to introduce 
such a bill during the 2018 legislative session,”? and will be continuing to advance competency-based 
education legislation in 2019. 


In fact, explicitly authorizing competency-based education would be an ideal way for the state to support 
and encourage the process of local school and district learning communities defining and measuring their 
own student-centered outcomes—which is the focus of the ideas in the next section. 


In 2015, New Hampshire received a federal waiver to use performance assessments—in some grade 
levels and subjects—to make proficiency and growth determinations for students as part of their 
statewide testing and accountability system. Students still take standardized tests in some grades; 


for example, they take the state’s standard math test in grades 4 and 8, and the SAT in grade 11. 


New Hampshire kicked off this transition with a pilot among a small group districts, and has been 
adding districts every year, with the idea that eventually the new system will be used statewide. 
Currently, over 30 percent of districts in the state participate in PACE. Educators who participate 
go through extensive training on performance assessments, and regular calibration of inter-rater 
reliability. Early data from districts participating in the pilot show promising academic results.®° 


educationevolving.org 


B. Ideas for School and District 
Learning Communities 


Define your local community’s target outcomes, 
i.e. “portrait of a graduate” 


Imagine if, in every learning community across the state, students, 
families, educators, and community members came together to 
explicitly define a “portrait of a graduate”, which articulated the 
learning outcomes that mattered to them. The various outcomes 
described above in Part 71 are intended to be a resource for 
educators who are facilitating the process, to help align what they 
are hearing from their communities and families with outcomes 
that have evidence in research. 


Imagine if, coming out of this process, the learning community 
produced a set of, say, 5 to 10 key competencies or outcomes they 
valued, depicted in some graphic like the one shown in Figure 3. 
For additional examples and resources for facilitating this process, 
see: portraitofagraduate.org. 


And finally imagine if, as a result of the collaborative process 
through which they were conceived, these outcomes were not just 
a poster on the wall, but were truly shared, understood, valued, 
and lived out by all members of the learning community and 
were palpable everywhere—from the objectives in courses and 
curriculum, to the conversations among educators in PLCs, to the 
grading categories used in online learning management systems, 
to the report cards sent home with students. 


Saint Peter 
Public Schools 


Graduate Portrait 


Creativity & 
Innovation 
Saint Peter Public Schools 
is committed to developing 
career, college, and life readiness 
skills in our PreK-12 students to Well-being Content 
prepare them for their future. & Resilience Mastery 
PUBL, 
3“ °. ; 
% (oellleloronrelivon 


Figure 3. A sample "portrait of a graduate", used with permission from 
Saint Peter Public Schools, a district of about 2000 students located in 
south central Minnesota. 
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(2) Implement software for documenting evidence and aggregating multiple measures 


Imagine if schools and districts used 
online software in which students 
could document their evidence of 
progress toward learning outcomes. The 


evidence logged in that system could ne if this collected evidence and data 
take many forms—scores on state and jars i re : 

local assessments, student surveys, Wasn tjust used to “check a box” or givea 
feedback from teachers, pictures of grade, but became the basis for ongoing 
students’ projects, videos of students’ : : 

preseniatioiie, wesults on (naifermence reflection and conversation... about the 


assessments, and more. The system next step in each student’s learning j 


could aggregate these pieces of evidence 
and indicate a student’s overall progress 
toward mastering both the outcomes the 
community had articulated (per Idea B1) 
as well as the state standards. 


And, imagine if this collected evidence and data wasn't just used to “check a box” or give a grade, but 
became the basis for ongoing reflection and conversation among students, teachers, and families, about 
the next step in each student’s learning journey. 


Several software vendors offer platforms that can serve this function, and the landscape of promising 
options is always growing. In the years ahead, it’s critical that these vendors safeguard the privacy of 
student data collected, and provide opportunities for all schools and districts, including those that are 
under-resourced, to access this emerging technology. 


© Join with other learning communities to share lessons learned 


Imagine if educators from several different schools and districts, as well researchers and measurement 
experts, came together as a “collaborative” to strengthen their practice of defining and measuring student- 
centered outcomes. For example, imagine if such a collaborative hosted a workshop on the competency 
of “critical thinking”, at which educators and researchers shared their definitions of the concept, discussed 
actual samples of student work, created performance tasks and rubrics to assess it, and refined strategies 
for helping students to improve on this skill. 


¢ G... if educators from several 


different schools and districts, as well 
researchers and measurement experts, 


came together as a aa by 
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@ Create a dashboard or scorecard for local learning community outcomes 


Imagine if schools and districts created their own dashboard or scorecard, built from their locally-defined 
learning outcomes per Idea B1 and other school outcomes, and used that dashboard both to reflect on 
and improve their programs and initiatives, and also to communicate transparently with families and 
communities about their progress.®' And, imagine if their relationship with the entity that supports and 
oversees them—be it a school board for districts, or charter authorizers and boards for charter schools— 
used measures from this dashboard as the basis for support and accountability. 


(GOF-TT-s) ab lo hisan batokcr- hia Op ahbuk-io mclol alololm Dy iia leit 


Lindsay Unified School District, located Southeast of Fresno, California, serves 4,200 students, nearly 
all of whom qualify for free or reduced price lunch, and nearly half of whom are English language 
learners. In 2007, after an extensive community engagement process, the district settled on a bold 
vision for competency-based learning, centered on 10 core values.® 


Students’ learning is oriented around learning targets, which include both reading and math standards 
as well as seven “lifelong learning” standards. Teachers provide various forms of customized support 
to students, including small-group instruction and one-on-one coaching, while other students work 
independently. Students have several options for demonstrating evidence of having met a learning 
target, including projects, performance tasks, formative assessments, and more.®? 


Since implementing their new model, the district has seen gains in reading, math, and science 
proficiency rates, as well as substantial improvements on school culture related measures such as 
suspension rates.** 


G@r-Ti-es) ab leh! sa hho saovel aloo] mcoral,<-1ero}cenhalemaWaes 


High School for Recording Arts is a small charter high school in Saint Paul, Minnesota that uses a 
hip-hop themed, project-based learning program to engage students, many of whom are behind 
on credits or have previously dropped out. The school has defined 6 competencies as a community, 
including social justice, citizenship, and community involvement; self-starter and self-advocacy skills; 
entrepreneurialism and career readiness; creativity and critical thinking; communication and literacy; 
inquiry and technology skills; and practical life skills. Each competency contains a number of specific 
learning objectives students must meet within it. 


Students complete projects and take courses both to master local competencies and learning 
objectives, and also to earn credits and meet standards as required by the state. A single project 
or course usually helps students meet both local outcomes and state standards at the same time. 
Students and teachers use an online system to help document and track progress, and to generate 
transcripts. 


Additionally, students take the Hope Survey,> which measures a number of social-emotional 
competencies, as well as the NWEA MAP test to track progress and identify areas for improvement in 
language arts and math. Finally, HSRA tracks a number of schoolwide measures, including the overall 
rate of credit attainment among students, attendance and graduation rates, and more. 
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C. Ideas for Individual Students and Their Families 
fcr) Families make informed choices, and become advocates, based on better information 


Imagine if each student sat down with their family, or other 
supportive adults in their lives, to select a quality school 


that was a good fit for them. This might entail browsing the agine if the teacher and 
improved state “report card” website (see Idea A2), as well as : 
browsing the websites and dashboards of individual schools student then together decided 
to see their learning program themes and the outcomes they how required state standards 
value (see Idea B1 and B4). Imagine if students and families then . : 

used that information not only to inform their school selection, a nd lea ring community 

but also to raise helpful, informed questions about how their outcomes would be met, while 


school might improve. 


Each student creates a personal learning plan for 
meeting state, learning community, and 


individual outcomes desired out e 
Imagine if each student were asked by their teacher: “What's 
important to you?” Imagine if the teacher and student then 


together decided how required state standards and learning 

community outcomes would be met, while also respecting the students’ own interests, aspirations, 
and desired outcomes. Imagine if these conversations were captured in a personal learning plan for 
each student, which plotted their educational journey toward meeting state standards, local learning 
community outcomes, and their own personal learning outcomes. 


Conclusion 


We close by reiterating a point from the introduction. It’s imperative we value an expanded set of student- 
centered outcomes that will equitably prepare all students for success in the 21st century. At the same 
time, we must continue to value academic achievement, which research shows is also critical for student 
success. Debates that put one above 
the other are not productive; there is 
strong evidence we must value both. 


Research on social change suggests 
this needed shift toward “valuing both” 
willbe most authentic, deep, andlasting 
if it begins with those who are ready.®° 


Fortunately, pioneering educators - Dr. Jeffrey Duncan-Andrade and Dr. Ernest Morrell®® 


and policymakers are already leading 
the way. We envision the ideas presented in this paper will be embraced by increasing numbers of early 
adopters, with additional students, families, educators, communities, schools, districts, and states coming 
on board as they are ready.®’ We are here to support those early adopters however we can. 


Insum, to equitably prepare all students for success in a rapidly changing world, schools must be redesigned 
with students at the center. Partly this will mean changing students’ learning experiences, but it must also 
involve changing the outcomes we define and measure. The two must go hand in hand; to flip the old 
adage around, we cannot change what’s done if we don’t define and measure what matters. 
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also respecting the students’ 
own interests, aspirations, and 


Appendix A: Research Methodology 


Interviews 

We conducted over 50 interviews with students, families, educators, policymakers, community advocates, 
researchers, and business owners. We sought to speak with a diverse set of individuals, in terms of their 
geography, cultural background, race, ethnicity, sector, and political stance. 


Literature Review 

We reviewed an extensive collection of reports and peer-reviewed journal articles in preparing Part 1 of this 
paper. To comprehensively describe the results of that literature review in this paper would have expanded 
greatly its scope, and detracted from its primary purpose, which is to present a vision and ideas for the 
future. For those interested, we have compiled a full bibliography of the sources from which we drew our 
conclusions, at: http://www.educationevolving.org/content/student-centered-outcomes-bibliography 


(Appendix continued on the following page.) 
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Frameworks Reviewed 

Part of our literature review was to consult the “outcomes frameworks” that other researchers and 
organizations have compiled, in seeking to answer similar research questions to those posed in this paper. 
The following is a full set of those frameworks which we reviewed and analyzed as part of our research. 
We humbly acknowledge the trailblazing work the authors of these frameworks have done. 


Publisher Mitte of Framework or Top-Level Outcome Domains 
Report 


aGal (Go) (=o) =i Self-awareness; Self-management; Social awareness; 

Competencies Relationship Skills; Responsible decision-making 
Center for Curriculum [ Four-Dimensional Knowledge; Skills; Character; Meta-Learning, i.e. learning 
Redesign (CCR) Education how to learn 


College and Career 
(@e)alatqved ie) Readiness: What Do 
VV em utetel ala 


; College and Career Cognitive strategies; Content knowledge; Learning skills 
Se SUE eReiil 3) Readiness Framework {and techniques; Transition skills and knowledge 


Mastering rigorous academic content; Learning 
Deeper Learning how to think critically and solve problems; Working 
Framework collaboratively; Communicating effectively; Directing 
one’s own learning; and Developing an academic mindset 


Knowledge; Skills; Productive dispositions and behaviors; 
Educational, career, and civic engagement 


Hewlett Foundation 


Knowledge (i.e. content, transfer); Skills (including higher 
(eo) cel=Jmnaalial.<jaloeme)e-lalali green =anen uD) i<j oles<}1a(0) aia (ale ducel i ayes 
social-emotional skills and behaviors) 


KIPP Character Zest; Grit; Optimism; Self-control; Gratitude; Social 
KIPP : : ie 
Strengths intelligence; Curiosity 


Four Principles 
stanjoyeyel(=xeNlanualc 
INGLE WA nals er- |e Gli nels) 
of Courage 


Next Generation Mave Habits of success; Creative know how; Content 
Learning Challenges yay. knowledge; Wayfinding abilities 


Behaviors related to conscientiousness; Sense of 
Supporting Students’ | belonging; Academic self-efficacy; Growth mindset; 
College Success Utility goals and values; Intrinsic goals and interest; 

Prosocial goals and values; Positive future self 


National Research Education for Life and | Cognitive skills; Interpersonal competencies; 
(@olvlarellt NAVfo) a4 Intrapersonal competencies 


i a\=mer-a Gsm (ol di (er=|Gualinl.diay-emee)nalanlelal(er-nd(ela mero) ic-|ele)e-lule)an 
creativity); Life and career skills; Information, media, and 
technology skills; Key subjects and 21st century themes 


Innovation Lab Knowledge, Skills, and 
Network Dispositions 


Dr. Martin Brokenleg Belonging; Mastery; Independence; Generosity 


IN Fe)ele) pt) Wy Vere lel=ia al (=ss3 


Partnership for 21st Framework for 21st 
Century Skills Century Learning 


Motivation; Effort; Self-regulated learning; Self-efficacy; 
Academic self-concept; Antisocial and prosocial behavior; 
Coping and resilience 


: : ; F iC iti kills; Content k ledge; Habits of : 
S10] nalnalion -10]o)i(emsve alelelicm usiel(-lalecmolmelUlaalaalie Pee gee ort are aa ae eee 
Sense of purpose 


Masters of fundamental literacies; Holders of 
XQ Learner Goals foundational knowledge; Original thinkers; Generous 
collaborators; Learners for life 


Noncognitive Skills in 
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